Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate a native executable via GraalVM #4 #11

Merged
merged 4 commits into from
Apr 13, 2019
Merged

Generate a native executable via GraalVM #4 #11

merged 4 commits into from
Apr 13, 2019

Conversation

nicokosi
Copy link
Owner

@nicokosi nicokosi commented Feb 7, 2019

Implement #4.

@nicokosi
Copy link
Owner Author

nicokosi commented Feb 26, 2019

Trial n°1, fails with Non-reducible loop:

The native executable cannot be generated, visibly because of oracle/graal#366:

Build nicokosi/ipullpitoK's executable from JAR via GraalVM:
1.0.0-rc12: Pulling from oracle/graalvm-ce
Digest: sha256:7b41bbd77be7febbc696a308438b0eeb3464d3417459d9ebc29db5a7668580b7
Status: Image is up to date for oracle/graalvm-ce:1.0.0-rc12
Build on Server(pid: 10, port: 33733)*
[pullpitoK:10]    classlist:   1,458.82 ms
[pullpitoK:10]        (cap):     622.06 ms
[pullpitoK:10]        setup:   1,545.73 ms
warning: unknown locality of class Lkotlin/coroutines/CoroutineContext$plus$1;, assuming class is not local. To remove the warning report an issue to the library or language author. The issue is caused by Lkotlin/coroutines/CoroutineContext$plus$1; which is not following the naming convention.
[pullpitoK:10]     analysis:   3,349.03 ms
Error: Non-reducible loop
Detailed message:
Error: Non-reducible loop
Call path from entry point to pullpitok.AppKt$main$1.invokeSuspend(Object): 
	at pullpitok.AppKt$main$1.invokeSuspend(App.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:285)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742)
	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:481)
	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
	at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)
Original exception that caused the problem: org.graalvm.compiler.core.common.PermanentBailoutException: Non-reducible loop
	at org.graalvm.compiler.java.BciBlockMapping.computeBlockOrder(BciBlockMapping.java:882)
	at org.graalvm.compiler.java.BciBlockMapping.build(BciBlockMapping.java:527)
	at org.graalvm.compiler.java.BciBlockMapping.create(BciBlockMapping.java:1109)
	at org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:806)
	at org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:784)
	at org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:95)
	at org.graalvm.compiler.phases.Phase.run(Phase.java:49)
	at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
	at org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
	at org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
	at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:205)
	at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:324)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107)
	at com.oracle.graal.pointsto.flow.SpecialInvokeTypeFlow.onObservedUpdate(InvokeTypeFlow.java:421)
	at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:347)
	at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:389)
	at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:508)
	at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:174)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

com.oracle.svm.core.util.UserError$UserException: Non-reducible loop
Detailed message:
Error: Non-reducible loop
Call path from entry point to pullpitok.AppKt$main$1.invokeSuspend(Object): 
	at pullpitok.AppKt$main$1.invokeSuspend(App.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:285)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742)
	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:481)
	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)
	at com.oracle.svm.core.code.IsolateEnterStub.PosixJavaThreads_pthreadStartRoutine_e1f4a8c0039f8337338252cd8734f63a79b5e3df(generated:0)
Original exception that caused the problem: org.graalvm.compiler.core.common.PermanentBailoutException: Non-reducible loop
	at org.graalvm.compiler.java.BciBlockMapping.computeBlockOrder(BciBlockMapping.java:882)
	at org.graalvm.compiler.java.BciBlockMapping.build(BciBlockMapping.java:527)
	at org.graalvm.compiler.java.BciBlockMapping.create(BciBlockMapping.java:1109)
	at org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:806)
	at org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:784)
	at org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:95)
	at org.graalvm.compiler.phases.Phase.run(Phase.java:49)
	at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
	at org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
	at org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
	at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:205)
	at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:324)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107)
	at com.oracle.graal.pointsto.flow.SpecialInvokeTypeFlow.onObservedUpdate(InvokeTypeFlow.java:421)
	at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:347)
	at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:389)
	at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:508)
	at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:174)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

	at com.oracle.svm.core.util.UserError.abort(UserError.java:67)
	at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:701)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:492)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:410)
	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Error: Processing image build request failed

@nicokosi
Copy link
Owner Author

nicokosi commented Feb 26, 2019

Trial n°2 with kraal, fails due to Non-instantiated type referenced by a compiled method: kotlin.Result$Companion:

Fatal error: com.oracle.svm.core.util.VMError$HostedError: Non-instantiated type referenced by a compiled method: kotlin.Result$Companion
	at com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:66)
	at com.oracle.svm.hosted.image.NativeImageCodeCache.addConstantToHeap(NativeImageCodeCache.java:158)
	at com.oracle.svm.hosted.image.NativeImageCodeCache.addConstantsToHeap(NativeImageCodeCache.java:148)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:585)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:410)
	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Error: Processing image build request failed

@nicokosi nicokosi force-pushed the graalvm branch 2 times, most recently from 276024d to c590137 Compare March 12, 2019 05:21
@nicokosi
Copy link
Owner Author

nicokosi commented Mar 14, 2019

Trial n°3 without Ktor/Kotlin Coroutines, fails at runtime with a java.lang.UnsatisfiedLinkError:

$ ./pullpitoK python/peps

WARNING: The sunec native library, required by the SunEC provider, could not be loaded. This library is usually shipped as part of the JDK and can be found under <JAVA_HOME>/jre/lib/<platform>/libsunec.so. It is loaded at run time via System.loadLibrary("sunec"), the first time services from SunEC are accessed. To use this provider's services the java.library.path system property needs to be set accordingly to point to a location that contains libsunec.so. Note that if java.library.path is not set it defaults to the current working directory.
Exception in thread "main" java.lang.UnsatisfiedLinkError: sun.security.ec.ECKeyPairGenerator.generateECKeyPair(I[B[B)[Ljava/lang/Object; [symbol: Java_sun_security_ec_ECKeyPairGenerator_generateECKeyPair or Java_sun_security_ec_ECKeyPairGenerator_generateECKeyPair__I_3B_3B]
	at com.oracle.svm.jni.access.JNINativeLinkage.getOrFindEntryPoint(JNINativeLinkage.java:145)
	at com.oracle.svm.jni.JNIGeneratedMethodSupport.nativeCallAddress(JNIGeneratedMethodSupport.java:54)
	at sun.security.ec.ECKeyPairGenerator.generateECKeyPair(ECKeyPairGenerator.java)
	at sun.security.ec.ECKeyPairGenerator.generateKeyPair(ECKeyPairGenerator.java:128)
	at java.security.KeyPairGenerator$Delegate.generateKeyPair(KeyPairGenerator.java:703)
	at sun.security.ssl.ECDHCrypt.<init>(ECDHCrypt.java:78)
	at sun.security.ssl.ClientHandshaker.serverKeyExchange(ClientHandshaker.java:783)
	at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:302)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
	at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
	at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
	at java.net.URL.openStream(URL.java:1045)
	at kotlin.io.TextStreamsKt.readBytes(ReadWrite.kt:150)
	at pullpitok.github.EventClient.githubEvents(EventClient.kt:9)
	at pullpitok.AppKt.main(App.kt:19)

Related resources:

@nicokosi nicokosi force-pushed the graalvm branch 2 times, most recently from 7e8ca6c to 283b305 Compare April 9, 2019 05:00
@nicokosi nicokosi removed the wip label Apr 9, 2019
@nicokosi
Copy link
Owner Author

nicokosi commented Apr 9, 2019

Trial n°4 (added the missing /jre/lib/*/libsunec* shared library):

$ ./generate-executable.sh                                  9:07  nkosinski@VID01971
Build nicokosi/pullpitoK JAR file:

BUILD SUCCESSFUL in 0s
2 actionable tasks: 2 up-to-date
Install GraalVM via SDKMAN!:

                                -+syyyyyyys:
                            `/yho:`       -yd.
                         `/yh/`             +m.
                       .oho.                 hy                          .`
                     .sh/`                   :N`                `-/o`  `+dyyo:.
                   .yh:`                     `M-          `-/osysoym  :hs` `-+sys:      hhyssssssssy+
                 .sh:`                       `N:          ms/-``  yy.yh-      -hy.    `.N-````````+N.
               `od/`                         `N-       -/oM-      ddd+`     `sd:     hNNm        -N:
              :do`                           .M.       dMMM-     `ms.      /d+`     `NMMs       `do
            .yy-                             :N`    ```mMMM.      -      -hy.       /MMM:       yh
          `+d+`           `:/oo/`       `-/osyh/ossssssdNMM`           .sh:         yMMN`      /m.
         -dh-           :ymNMMMMy  `-/shmNm-`:N/-.``   `.sN            /N-         `NMMy      .m/
       `oNs`          -hysosmMMMMydmNmds+-.:ohm           :             sd`        :MMM/      yy
      .hN+           /d:    -MMMmhs/-.`   .MMMh   .ss+-                 `yy`       sMMN`     :N.
     :mN/           `N/     `o/-`         :MMMo   +MMMN-         .`      `ds       mMMh      do
    /NN/            `N+....--:/+oooosooo+:sMMM:   hMMMM:        `my       .m+     -MMM+     :N.
   /NMo              -+ooooo+/:-....`...:+hNMN.  `NMMMd`        .MM/       -m:    oMMN.     hs
  -NMd`                                    :mm   -MMMm- .s/     -MMm.       /m-   mMMd     -N.
 `mMM/                                      .-   /MMh. -dMo     -MMMy        od. .MMMs..---yh
 +MMM.                                           sNo`.sNMM+     :MMMM/        sh`+MMMNmNm+++-
 mMMM-                                           /--ohmMMM+     :MMMMm.       `hyymmmdddo
 MMMMh.                  ````                  `-+yy/`yMMM/     :MMMMMy       -sm:.``..-:-.`
 dMMMMmo-.``````..-:/osyhddddho.           `+shdh+.   hMMM:     :MmMMMM/   ./yy/` `:sys+/+sh/
 .dMMMMMMmdddddmmNMMMNNNNNMMMMMs           sNdo-      dMMM-  `-/yd/MMMMm-:sy+.   :hs-      /N`
  `/ymNNNNNNNmmdys+/::----/dMMm:          +m-         mMMM+ohmo/.` sMMMMdo-    .om:       `sh
     `.-----+/.`       `.-+hh/`         `od.          NMMNmds/     `mmy:`     +mMy      `:yy.
           /moyso+//+ossso:.           .yy`          `dy+:`         ..       :MMMN+---/oys:
         /+m:  `.-:::-`               /d+                                    +MMMMMMMNh:`
        +MN/                        -yh.                                     `+hddhy+.
       /MM+                       .sh:
      :NMo                      -sh/
     -NMs                    `/yy:
    .NMy                  `:sh+.
   `mMm`               ./yds-
  `dMMMmyo:-.````.-:oymNy:`
  +NMMMMMMMMMMMMMMMMms:`
    -+shmNMMMNmdy+:`


                                                                 Now attempting installation...


Looking for a previous installation of SDKMAN...
SDKMAN found.

======================================================================================================
 You already have SDKMAN installed.
 SDKMAN was found at:

    /home/nkosinski/.sdkman

 Please consider running the following if you need to upgrade.

    $ sdk selfupdate force

======================================================================================================


Stop! java 1.0.0-rc-15-grl is already installed.

Using java version 1.0.0-rc-15-grl in this shell.
Copy 'libsunec' shared library (Sun Elliptic Curve crypto):
Build nicokosi/pullpitoK's executable from JAR via GraalVM:
[pullpitoK:27859]    classlist:   1,514.37 ms
[pullpitoK:27859]        (cap):     939.72 ms
[pullpitoK:27859]        setup:   2,223.83 ms
[pullpitoK:27859]   (typeflow):  10,693.98 ms
[pullpitoK:27859]    (objects):  10,319.45 ms
[pullpitoK:27859]   (features):     478.82 ms
[pullpitoK:27859]     analysis:  21,901.45 ms
[pullpitoK:27859]     universe:     667.78 ms
[pullpitoK:27859]      (parse):   1,876.36 ms
[pullpitoK:27859]     (inline):   2,805.87 ms
[pullpitoK:27859]    (compile):  15,609.01 ms
[pullpitoK:27859]      compile:  21,401.64 ms
[pullpitoK:27859]        image:   1,613.72 ms
[pullpitoK:27859]        write:     305.11 ms
[pullpitoK:27859]      [total]:  49,825.60 ms
 => Check the executable: 
Usage: pullpitoK <repository> <token>

A command line tool to display a summary of GitHub pull requests.

<repository>: a GitHub repository.
    Example: python/peps

<token>: an optional GitHub personal access token
\nExecutable has been generated in local directory, try it copy/pasting this command:
 $> ./pullpitoK python/peps

Executable can also be run from an other directory via a 'PULLPITOK_LIBSUNEC'
environment variable in order to load the 'libsunec' shared library (Sun
Elliptic Curve crypto) bundled in the Java Development Kit.
For instance, on a 64-bit Linux:
 $> PULLPITOK_LIBSUNEC=/home/nkosinski/.sdkman/candidates/java/1.0.0-rc-15-grl/jre/lib/amd64/ ./pullpitoK python/peps

then:

$ time ./pullpitoK --help                                   9:07  nkosinski@VID01971
Usage: pullpitoK <repository> <token>

A command line tool to display a summary of GitHub pull requests.

<repository>: a GitHub repository.
    Example: python/peps

<token>: an optional GitHub personal access token
./pullpitoK --help  0.00s user 0.00s system 88% cpu 0.002 total

and:

PULLPITOK_LIBSUNEC=/home/nkosinski/.sdkman/candidates/java/1.0.0-rc-15-grl/jre/lib/amd64 ./pullpitoK python/peps




$ PULLPITOK_LIBSUNEC=/home/nkosinski/.sdkman/candidates/java/1.0.0-rc-15-grl/jre/lib/amd64 ./pullpitoK python/peps
pull requests for "python/peps" ->

	opened per author
		brettcannon: 1
		ncoghlan: 1
		gvanrossum: 4
		markshannon: 3
		jdemeyer: 2
		BTaskaya: 1
		pablogsal: 2
		willingc: 1
		vstinner: 1
		ericvw: 1
		yaroslav-semeniuk: 1
		jimbo1qaz: 1
		pganssle: 1
		zooba: 2
		RazerM: 1
		mcfarke311: 1
		uranusjr: 1
		JukkaL: 1

	commented per author
		brettcannon: 8
		ncoghlan: 2
		gvanrossum: 6
		markshannon: 1
		jdemeyer: 7
		ilevkivskyi: 1
		pablogsal: 5
		willingc: 8
		vstinner: 4
		ericvw: 8
		pganssle: 4
		encukou: 5
		mariocj89: 3
		uranusjr: 1
		pfmoore: 1
		JelleZijlstra: 7

	closed per author
		brettcannon: 5
		ncoghlan: 3
		gvanrossum: 10
		markshannon: 1
		ilevkivskyi: 1
		pablogsal: 4
		Rosuav: 1
		zooba: 2
		Mariatta: 1

@nicokosi nicokosi force-pushed the graalvm branch 2 times, most recently from 4fd6fec to 168a051 Compare April 10, 2019 11:39
Install GraalVM (https://www.graalvm.org/) via SDKMAN! (https://sdkman.io/).

GraalVM is used to build a native image (executable) from the fat JAR.

Note: 'libsunec' native library (Sun Elliptic Curve crypto) needs to be in java.library.path, it is copied in current directory).
@nicokosi nicokosi merged commit 55e79dd into master Apr 13, 2019
@nicokosi nicokosi deleted the graalvm branch April 13, 2019 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant